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MINIMAX  PROCEDURES  TOR  TO -VALUED  DECISION 
■  PROBLEMS  rilETi  THE  SIZE  OF  SAMPLE  IS  FIXED* 

by 

S.  G.  Allen,  Jr. 


1.  The  Mini max  Solution  for  the  Cage  of  a  Simple  Dichotomy. 

The  problem  to  be  considered  is  a  statistical  decision  procedure  for 
choosing  between  two  alternative  actions,  A^  and  A 2,  after  taking 
n  Independent  observations  on  a  random  variable  x.  The  probability 
density  p(x,©)  of  x  Is  known  except  for  the  value  of  a  real  para¬ 
meter  0  which  Is  assumed  to  be  one-dimensional*  1/  The  parameter 

and  -Ag  such  that 

decision  Ai  Is  preferred  if  Oe-A-^  and  Ag  preferred  if  Ot-A-g.  The 
co 8 1 s  of  decisions  A^  and  Ag  are,  respectively. 


space  -A-  is  partitioned  into  two  subspaces-A, 


1.1 

and 


n  0, 
>0, 


Of/lg 


1.2  Wg(9) 


£  o,  9e^1 

£  0,  9£-A.g  « 


The  motivating  idea  for  Theorem  A  of  this  report  Is  to  be  found 
In  a  lot  acceptance  sampling  procedure  proposed  in  an  unpublished 
paper  by  Mr.  Norman  Rudy  of  Sacramento  State  College.  Discussion 
with  members  of  the  Department  of  Statistics,  Stanford  University, 
in  particular  with  Professor  M.  A.  Girshlck,was  most  beneficial 
In  the  formulation  of  the  present  draft  of  the  reoort. 

*  The  main  results  of  this  paper  will  be  based  on  the  assumption 
that  the  probability  distribution  of  x  is  continuous.  The  necessary 
modification  of  the  results  for  the  case  of  discrete  probabilitv 
distributions  will  be  discussed  in  a  concluding  section  of  the 
paper. 
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I f  the  parameter  space  consists  of  a  simple  dichotomy 
(l.e.  A  consists  of  the  single  noint  and -fig,  the  single 
point  ©gif  the  minimax  procedure  is  well  known.  The  relevant 
statistic  is  the  likelihood  ratio 

n 

1.3  .  a-  ^(9i.0S) 

-  TT  p(x1,91)  v- 

isl 

and  the  decision  procedure  which  minimizes  the  maximum  expected 
cost  is  to 

1.4  choose  A^f  if  *  £  c,  and 

1.5  choose  A2,  if  >  c, 

where  the  criterion  c  satisfies  the  relation 


2»  The  Minimax  Solution  to  a  More  General  Problem*  The 
assumption  that  and  fig  «®cb  consists  of  a  single  point  is  often 
a  very  artificial  one.  A  more  general  formulation  of  the  two-valued 
decision  problem  is  the  following:  SL  is  an  interval,  ©£-fl^  if 

an<3  where  the  costs  of  decisions  A^  and  Ag 


3- 


ar©  Riven  by  1.1,  1.2,  and 


2.1  w1(0o)  -  v2(0o)  s  0.  S/ 


This  statement  of  the  problem  avoids  the  often  unrealistic  postulate 
of  an  "indifference  interval",  i.e.  an  Interval  such  that  decisions 
A^  and  Ag  are  either  costless  or  equally  co3tly  if  0  assumes  a 
value  in  this  interval.  However,  in  thi3  more  general  formulation, 
is  the  likelihood  ratio  test  for  some  ^l  an^  ®2£/12  R 

minimax  decision  procedure?  .  •  . 

The  following  theo  re rri'  s upplie  s  th e  answer. 

Theo  re m  A .  Let  e-&b. ■■ ••  • ;  ■ 


2.2  R1(c,01,02*°)  ”  Wj/0)  Px*£a(©1»02)^  c IqJ  and 

2.3  H2( c,0^,02j0)  s  w2(0)  Pr£^(01,92)>  cjOJ  ,  where 

and  92t*n,2“  71100  the  likelihood  ratio  test  is  a  minimax 

procedure  if  and  only  if  there  exists  a  triple  (c0,©°,©2)  such  that 

2.4  gajc  Ritco.eJ.sg.e)  =  R1(e0,e£,Gl’,e°)  -  R2(co,e°,9§,0|) 

-  oax  RB(en,9?,92*9)  » 

Proof  of  necessity: 

Suppose  the  likelihood  ratio  test  for  some  ®l*,nV  ®2*/1’2» 
and  c  r  c°  is  minimax.  Then  2.4  follows. 


y 


It  seems  only  reasonable  to  assume  that  inf  w.  (c)  g  wi(©0)  And 

A*  A  *  * 


©t-h, 


inf  w2(0)  -  w2^®o^“  With  this  in  view,  there  is  no  loss  in 
generality  in  requiring  2.1, 


4 


Proof  of  sufficiency: 

Let  F0  denote  the  likelihood  ratio  toot  procedure  associated 
with  the  triple  (c0,©£,©Sp  satisfying  2.4.  Then  tho  expected  cost 
of  this  procedure  is  given  by 

2.5  R(F0,G)  -  / w1(0)  cQf 0,7  dO(O) 

^2 

+  j[v2( 9)  Pr[^0O,Q|)>  coj0j.  d0(9) 

where  dG(G)  denotes  any  a  priori  probability  measure  overdo  Since 

Rl(co»®i*®2»®2^  and  R2^ coj9i’®2,{31^  are  the  maxima  of  integrands 
appearing  in  2.5,  it  follows  that 

2.6  R(F0,G)|  /r1(co,0|,0|,O|5  dG(9)  +  j.  Rg(  c0,©° Og toJ)  dG(O) 

2  .1 

”  co*®l>®2*®2^  "  r2^  co  >9I*®2»®l)  0 

If  G  is  a  distribution  concentrating  all  probability  at  6°  and  0g  p 
2,6  is  a  strict  equality.  Therefore 

2.7  max  RCF0,G)  B  Ri(co,oj,©g,0§)  -  Rg(c0»°l»92*®l)'> 

Lot  Gq  be  the  a  priori  distribution  given  by 

2.8  g  -  Pr(9  a  «£} 

2.9  1-g  «  Pr(G  -  ©|) 


where  g  satisfies 


There  fore 


2.10 


co  s 


w2( e°)s 

^(egxi-c) 


Clearly  PQ  Is  the  Bayes  procedure  against  0Q. 

2.11  min  R(F,0_ )  =  R(Fo»0o)  » 

F 

where  P  la  any  decision  procedure  for  choosing  between  and  Ag. 
Since  GQ  Is  a  distribution  for  which  2.6  is  a  strict  equality,  the 
minimax  property  of  PQ  follows* 

3.  Assumptions  Under  Which  the  Likelihood  Ratio  Test  Is  a 
Mlnlmax  Procedure.,  It  is  now  of  interest  to  examine  a  class 
of  distribution  functions  and  a  class  of  cost  functions  for 

which  condition  2.4  of  Theorem  A  is  satisfied. 

Consider  a  family  of  probability  measures  defined  by 

3.1  dL  (sJO)  s  -1—.  e9*  df0(x)  , 

" 

where  f0(x)  is  a  measure  defined  over  an  interval  X,  and  where 


3.2  w  (©)  s  /°°  Q9x  d ro(x) 


is  a  bounded  function  of  8  defined  over  an  interval  Sly 5/  If  u  Is 
the  sum  of  n  Independent  random  variables  each  distributed  according 
to  3.1,  then  u  Is  distributed  according  to 


2/  The  properties  of  this  family  of  distribution  functions  are  given 
an  extensive  discussion  in  Section  3  of  Girshiek  and  Savage,  "Bayes 
and  Minimax  Estimates  for  Quadratic  Loss  Functions”,  Technical 
Report  Ho.  5,  Contract  Ho.  N6-0NR-251  Task  III,  Department  of 
Statistics,  Stanford  "University,  November  21,  1950? 


3.3  ar,(u|e) 


«  s8u  artu)  . 

This  Is  again  the  distribution  family  introduced  In  3-1  and  3.2, 
withYtu)  defined  over  an  Interval  U. 

In  view  of  the  form  of  the  distributions  3,1  and  3.3,  it  la 
simple  to  show  that  fbr  every  number  c  such  that  ^  ^  c  (where  Pi  is 
defined  in  1.3)  there  exists  a  unique  number  k  such  that  u  ^  k. 
This  correspondence  permits  the  functions  2.2  and  2.3  to  be 
rewritten  as 


3.4  R^k,©)  -  w1(9)  L(kje) 

3.5  R2(k,G)  =  w2(0)  [l-L(k)9)J  . 

These  functions  have  the  following  Important  property* 

Lemma  1. 

If  max  R.,(lc,©)  and  max  R?(k,9)  exist,  then  they  are,  resoectively „ 

Oe-^2  *  9cnl 

mono  tonic  Increasing  and  decreasing  functions  of  k. 

Proof: 

Let  max  R->  (k,9)  =  w.(9*)L(k|9»)  ,  ©'e-A* 

9**2  1  1  2 

and  max  RP(k,9)  -  w2(9*,)[l-L(k}Cn)]  ,  ©"S-A*,  « 


Obviously  L(k/0)  is  an  increasing  function  of  k  for  any  9. 
Thus  for  A  positive 


Wt(9»)  L(k|©*)4  w,(0»)  L(k+A|9’)<  max  w,(9)  L(k+A|9)«  max  R,(k+A,9) 

X  ©CAg  A  OlAg  X 

and 

*2(9")n-Ml:|0”Ut»2(0")fl-l.(lc-*|0"U|  max  w2(9)[l-Ltk-a|9")D 


max  Ro(k-A  ©) 

-  ^ 


as  desired* 
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It  Is  now  possible  to  prove  the  following 

Theorem  B.  Let  Ytu),  w2(9),  and  w2(0)  be  continuous  throughout 

4/ 

any  finite  intervals  In  IT  r(afb),Jlg9  and-A^#  respectively Let 

5,6  lim  R..(k,9)  2  lim  Ro(k,9)  u  0 
k*»a  1  k-frb 

uniformly  in  9.  Then  if  the  max  R,(kp9)  and  max  R2(k,9)  exist 

gtSl  L  9«n  * 

2  1 

for  some  kcU,  there  exists  a  unique  value  of  k*  II  3uch  that 


3.7 


max  R-i(kfO) 
**8 


max  Ro(k,9)  • 

9,al 


Proof: 

Suppose  for  some  k  2  k_E.II,  max  R..  (k^,©)  >  max  RofkQjO).  For 

0  QlAg  x  Qt/^ 

k<kQ,  Rn(k,9)  is  uniformly  bounded  by  max  Rl(ko,0)o  Since  R1(k,0) 

QeAg 

la  continuous  in  k  and  9,  the  max  R,  (k,0)  exists  and  Is  continuous 

9IA2 

for  k<k0.  Certainly  the  sup  Rg(k,9)  is  continuous  In  k,  and  for 


k  <  k_  sup  Ro( kvQ)  >  max  R2(k_,9)o  Since  by  3.6  max  R2(k,9)-*0  as 
k-?a,  there  must  be  some  k£  (a,k_)  such  that  max  R-j(k,9)  2  sup  Rofk,©), 

QEAg  9l,nl 

or,  actually,  max  R«.(k,9)  =  max  R2(k,0).  Furthermore  in  view  of 
Of/lg  gt£\ 

Lemma  1  there  Is  only  one  such  value  of  k.  If  on  the  other  hand 

max  Ro(k,0)  >max  R.fk,©)  for  seme  k  cU,  then.  In  like  manner,  the 
Qi^  *  9*^  A 

restriction  3.6,  Lemma  1,  and  the  continuity  hypothesis  imply  the 

existence  of  a  unique  kt(k  ,b)  such  that  max  R-.(k,0)  =  max  R«(k,9)e 

9CA2  A  0*^  * 


The  interval  XJ  may  of  course  be  the  open  interval  (-00,  00  )o 
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Co rollary  1.  LetY(u)„  w^(G),  and  Wp(0)  be  continuous  and  bounded 
over  U  -  (a,b),>^2,  and-f^,  resnectively-  5/ 

Then  there  exists  a 

unique  k  £  U  such  that  max  R-»(k,9)  -  max  Rp(k,9)<> 


Corollary  2.  Let 


w -^(9)  -  0(e  ) 


-S^Q 

3.9  w2(0)  -  0(e  A  ) 


^2  >° 
^  >0 


be  continuous  functions  throughout  any  finite  intervals  in/lg 

and-fl^,  respectively .  Let  Y(u)  be  continuous,  positive  ,  and 

less  than  Y(co)  for  any  real  u.  Then  there  exists  a  unique  k£U 

such  that  max  R;i(k,9)  -  max  R0(k,9). 

9<A2  ”  91/^  * 

Proof: 

For  A  >0, 


(1) 


11m  w1(9)L(k|9)^  lim  e 
9-*oo  9«»oo 


f  e0u  dYtu) 


/  ° 

-00 


d^(u) 


constant 


.k  (u+$2)0 

/  e  2  d?(u) 

4  11m  I® _ 

*  ®*»  /eeu  dYTu) 


.  constant 


k+^2+A 


(S2+k)o  /  dT(u) 

£  llm  "viirTS1  -  •  constant  -  0.  and 

"  9*oo  (J2^4k)9  oo 

e  J  dytn) 

k+«2+A 


f(u)  Is  of  course  always  bounded., 


9 


-  0  . 

Since 

(3)  lira  w1(©)L(k|0)  •  lim  w2(9)  [l-L(k|©)J  2  ^(6  )L(kJ©0) 

9^ao  ©♦-oo 

;  Wg(8o)fl-L(k|0o)J  =  0 


for  any  k,  the  max  R. { k#9)  and  max  Ro(k,0)  exist  !br  any  k. 

Qt\  1  9 tJ\  c 

The  min  <*>(©)  and  min«>(9)  are  positive  constants.  Therefore, 
9lAg  ©C^ 

for  9£/ig, 


10 


Sp9  />k  q., 

(4)  WjfOjLfkJO)  ^  e  J  ev  dYCu).  constant 

-oo 

,c  (f2*u)(0-©o«-l)  (i2+u)(0o-l) 

e  « 


dYCu).  constant 


U 

<  (£p+k)  (8-0.+1)  «k  (<T2+u)(0  -1) 

-  e  2  0  J  e  d  °  dY(u)  »  constant 


4  o 


*o+k  /’k  V, 

J  e  dY(u)  *•  constant, 

-oo 


and  similarly,  fbr  OtA^  g 

k-^i  co  (u-£»KQ0+l) 

(5)  wo(0)[l-L(kj0)]  ^  e  /  e  dVTu)  «  constant, 

2  k 


The  inequalities  (4)  and  (5)  demonstrate  that  3.6  of  Theorem  B 
is  satisfied.  The  conclusion  of  Corollary  2  follows. 

Theorem  B  serves  primarily  as  a  criterion  for  determining 
whether  or  not  mlnlmax  likelihood  ratio  test3  exist  in  particular 
cases.  The  corollaries  to  Ifteorem  B  are  interesting  as  applications 
of  this  criterion  in  the  analysis  of  certain  classes  of  sampling 
distributions  and  cost  functions.  Individual  cases  could  of  course 
be  subjected  to  a  much  more  direct  analysis. 

4.  Remarks  on  the  Discrete  Case.  The  generality  of  the 
family  of  distributions  Introduced  In  3*1  should  not  be  overlooked: 
it  includes  many  of  the  most  important  distributions  encountered 
in  statistics,  such  a3  the  normal,  binomial,  and  Poisson  distribu¬ 
tions.  Suopo3e  the  distribution  under  consideration  In  this  class 


-Il¬ 


ia  a  discrete  one,  and  suodoso  that  T*(u)  assumes  Jumps  at  each 

value  of  a  denumerable  sequence  In  w-  ich  the  values  are  placed 

in  order  of  Increasing  magnitude.  For  example,  in  the  binomial 

distribution,  u  =  0,1,2,...  ,n.  In  general  it  will  not  be  nossible 

to  find  a  value  of  k  in  such  a  sequence  so  that  max  Rn(k,0)  -  max  Rg(k,0) 

A  9*/^ 

However,  provided  that  all  other  conditions  of  Theorem  B  are 
satisfied  except  for  the  continuity  of  Y(u),  it  will  be  possible 
to  find  a  pair  (kQ,  kg)  such  that 

4.1  max  Ri{k  0)  <  max  R^Ck.,©) 

At  0  A  f  A  6  v 


91*2 


0^ 


max  Ri(k*, 0)>  max  Rp(^*®5  , 

A  *  O  Atf  A  ^  V 


where  kQ  is  the  next  larger  value  In  the  sequence  than  k0.  The 
minimax  procedure  Is  then  a  convex  linear  mixture  of  two  procedures 
involving  ICq  and  k^,  a  mixture  determined  by 

4.3  max  £fR_(ko,0}  +  (l-f)R,(k’  ©3  =  max  £fR2(k_,©)  +  (l-f)Ra(k*  ©Q, 
QCfl^  1  A  ©^ 


where  0  <  f  <■  1 
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